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Abstract 



We study Okada's conjecture on (q, t)-hook formula of general d-complete posets. Proctor 
classified d-complete posets into 15 irreducible ones. We try to give a case-by-case proof of 
Okada's (q, t)-hook formula conjecture using the symmetric functions. Here we give a proof of 
the conjecture for birds and banners in which we use Gasper's identity for VWP-series 12VK11. 



O ■ l Introduction and the main results 

(-h The aim of this paper is to prove Okada's multivariate hook formula conjecture for birds and 

banners, i.e., Theorem 11.91 His conjecture is for general d-complete posets, and here we give 
a partial proof for birds and banners only. Proctor [5] has classified d-completc posets into 15 
irreducible classes. Okada [5] has made his conjecture for general d-complctc posets. he has proven 
two cases in his paper and we settle two cases in this paper so that the rest 11 classes are still 
left. Even though we do a case-by-case proof, we need the Macdonald polynomials and Gasper's 
identity for very well-poised series 12 Wn in our proof. This paper is composed as follows. In this 
section we recall the fundamental conceits on d-complctc partitions, and then state our main result, 
i.e., Theorem ll.9l To sate Okada's conjecture we need the terminologies on d-complete posets. In 
Section[5]we recall the Macdonald polynomials. In Section[3]we rewrite Okada's conjecture by the 
Macdonald polynomials and use the fact that the Macdonald polynomials are basis for the ring of 
I the symmetric functions. In Section [4] we prove the Macdonald polynomial identities obtained in 

■ Section [3] using Gasper's identity. 

Let N (resp. Z) be the set of nonnegative integers (resp. integers). Throughout this paper we 
use the standard notation for q-series (see [TJ 121 HI [S] ) : 



{a;q)c 



(a ]q U = l\(l-a q % fa*)- - (ofl „. 9)oo 

for any integer n. Usually (a; q) n is called the q-shifted factorial, and we frequently use the compact 
notation: 

(01,02, .. . , o r ; q) n = (ai;q) n (a 2 ;q) n • • • (o r ; q) n . 
The r+i^r basic hypcrgcomctric scries is defined by 



Ol, <Z2, • • • , O r+ i \ ^ ( a l> a 2) • • • j fflr+l! l)n „ 

61,..., r J ^ {q,bi,...,b r ;q) n 
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A basic hypcrgcometric series r +i4>r is said to be balanced if it satisfies qa\ ■ ■ ■ a r+ i = b\ ■ ■ ■ b r and 

z = q, well-poised if it satisfies qa\ = a^b\ = • • • = a r+1 6 r , very well-poised if it is well-poised 

i i 

and satisfies b\ = a\ and b 2 = —a'l (see [31 §2.1]). If r+1 </v is very well-poised series, we use the 
notation 



r +iW r (ai, a 4 , 



r+Wr 



a\.qa^ .—qa^ ,a4,...,a T .-)_i 
l l 



9>z 



Proposition 1.1. Gasper's identity ([U p. 1065, (3.2)], [3J pp.250, Ex. 8. 15]) reads as follows: 



1 <?->:! 



■q q - 

bq/a,cq/a,dq/a' 'a 2 



x 12 Wu 




(a/d.bq/d, cq/d, abc/d; q) a 
(q/d, ab/d, ac/d, bcq/d; q) c 



bcq 
ad 



be 



ab ac q 
, — , — ,a, b, c;q, - 
d d a 



(1.2) 



where at least one of a, b, c is of the form q n (n = 0, 1, . . . ). 

We use the notation in [S]. For nonnegative integers n and m we write 

(t m+1 ;q) n 



and 



f(n;m) = f q ,t{n;m) 
F(x)=F(x;q,t) = 



(t m ;g)™ 
(fa;g)oo 



where g and t are parameters and x is a variable (see [SJ (5) (6)]). Hereafter we use the convention 
that f q ,t(n; m) = for a negative integer n < 0. 

We use the notation in H2] for partitions. Let A = (Ai,A2, ••.) be a partition, i.e., Ai > 
A2 > . . . with finitely many Ai unequal to zero. The length and weight of A, denoted by £(X) and 
A|, are the number and sum of the non-zero Ai respectively. When |A| = N we say that A is a 
partition of N, and the unique partition of zero is denoted by 0. The multiplicity of the part i in 
the partition A is denoted by mi(A). We identify a partition with its diagram (Ferrers graph) 



D(\) = {(i,j)e 



1<3< Ai}. 



(1.3) 



The conjugate A' of A is the partition obtained by reflecting the diagram of A in the main diagonal. 
A partition is said to be strict if we have strict inequalities Ai > A2 > • • • > A r > with r = ^(A). 
If A is a strict partition, then its shifted diagram is defined by 

5(A) = { g Z 2 : 1 < j < Ai + 1 - 1 }. (1.4) 

Hereafter we may use the same symbol A to represent its diagram (or shifted diagram). 



We use standard notation and terminology of [T^l Chapter 3] related to posets. We write x <• y 
if x is covered by y, i.e., x < y and there is no z E P such that x < z < y. A Hasse diagram is 
a diagram in which one represents each element of P as a vertex in the plane and draws an edge 
that goes upward from x to y whenever y covers x. 

Definition 1.2. ([H], [T21 §3.15]) Let P be a poset. A P-partition is a map ir : P — > N satisfying 

x < y in P ==>■ 7r(x) > 7r(y) in N. (1-5) 
Let srf (P) denote the set of P-partitions. 
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Figure 1: A double-tailed diamond poset (4(1) 

First, we review the definition and some properties of d-complete posets. (See [5J US]-) For 
k > 3, we denote by (4(1) the poset consisting of 2k — 2 elements, called double-tailed diamond 
poset, with the Hasse diagram depicted in Figure [TJ The two incomparable elements are called the 
sides, the k — 2 elements above them are called neck elements, and the maximum and minimum 
elements are called top and bottom respectively. If k = 3 then we call (fe(l) a diamond. Let P be 
a poset. An interval [w,v] — {x G P : w < x < v} is called a dk-interval if it is isomorphic to 
(4(1) A d~jl -interval (k > 4) is an interval isomorphic to (4(1) — {top}. A dj -interval consists of 
three elements x, y and w such that w is covered by both x and y. A poset P is d-complete if it 
satisfies the following three conditions for every k > 3: 

(Dl) If / is a djT -interval, then there exists an element v such that u covers the maximal elements 
of I and I U {v} is a (4-interval. 

(D2) If / = [w, v] is a (4-interval and the top v covers u in P, then it € /. 

(D3) There are no d^ -intervals which differ only in the minimal elements. 

We quote a proposition due to Proctor Proposition in §3] (also see [5J Proposition 4.1]): 

Proposition 1.3. ([SJ Proposition in §3]) Let P be a e?-complctc poset. Suppose that P is 
connected, i.e., the Hasse digram of P is connected. Then we have 

(a) P has a unique maximal element vq. 

(b) For each v G P, every saturated chain from v to the maximum element vq has the same 
length. 

Hence P admits a rank function r : P —> N such that r(x) = r(y) + 1 if x covers y. 

A rooted tree is a poset which has a unique maximal element, and is such that each non- 
maximal element is covered by exactly one other element. Let P be a poset with a unique maximal 
element. The top tree T of P is the filter (i.e., x € T and y > x implies y G T) of P, whose vertex 
set consists of all elements x G P such that every y > x is covered by at most one other element 
of P. T is clearly a rooted tree and an element of T is called top tree element. Afterwards we use 
a particular kind of rooted tree. Let / > and h > g > be integers. The rooted tree Y(f;g, h) 
consists of one branch element above which a chain of / elements has been adjoined and below 
which two non-adjacent chains with g and h elements, respectively. 

Let P be a connected d-complete poset with top tree T. An element x G P is said to be acyclic 
if x G T and it is not in the neck of any (4 -interval for any k > 3. An element of P is said to 
be cyclic if it is not acyclic. Let Q be a d-complete poset containing an acyclic element y. Let 
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P be a connected d-complete poset. By Proposition 11.31 (a), let x denote the unique maximal 
element of P. Then the slant sum of Q with P at y, denoted Q V \ X P, is the poset formed by 

creating a covering relation x <■ y. A d-complete poset P is slant irreducible if it is connected and 
it cannot be expressed as a slant sum of two non-empty d-complete posets. Suppose that P is a 

connected d-complete poset with top tree T. An edge x <■ y of P is a slant edge if x,y € T and 
y is acyclic. In [9] Proctor proves P is slant irreducible if and only if it contains no slant edges. 
Also, P is slant irreducible if and only if every acyclic element is a minimal element of its top 
tree. (0 Proposition C of §4]) Given any connected d-complete poset P, first locate all of its slant 
edges. These may be erased in any order to produce a collection Pi, P2,. . . of uniquely determined 
smaller non-adjacent connected d-complete posets. No new slant edges are created, and so each of 
Pi, P2,. . . are slant irreducible. We say that Pi, P2,. . . are the slant irreducible components of P. 
If P is an irreducible component, then its top tree T is of the form Y(f\ g, h) for some / > and 
h > g > 1 (0 Theorem of §5]). In the paper he establish the following theorem, which describe 
the structure of any connected d-complete poset. 

Proposition 1.4. (Proctor Theorem in §4]) Let P be a connected d-complete poset. It may 
be uniquely decomposed into a slant sum of one element posets and irreducible components. The 
top tree of P is an analogous slant sum of the top trees of the irreducible components. 

In §7 of [§] Proctor defines 15 disjoint classes of irreducible components Ifi,. .. ,^15 and have 
shown that these 15 disjoint classes exhaust the set of all irreducible components. For the list of 15 
classes of irreducible d-complete posets see [HI Table 1] . The diagram (|1.3[) of an ordinary partition 
A or the shifted diagram (|1.4j) of a shifted partition A is regarded as a poset by defining its order 
structure as 



(iiiii) > fe, J2) 



iy < i 2 and ji < j 2 - 



(1.6) 



By this order the poset represented by a diagram P = D(X) is called a shape with its top tree 
T = Y(f;g, h) where / = 0, g = £(X) and h = £(X')- We use ^1 to express the class of shapes 
which is a class of irreducible d-complete posets defined in [5]. 

Another important class ^2 is the set of posets P = 5(a) of shifted diagrams for strict partitions 
a, which is called shifted shapes with its top tree T = Y(f,g,h) where / = g = 1 and h = 1(a). 
Its Hassc diagram is designated by Figure [2] in which the first row has ai vertices, the second row 
ct2 vertices and so on. When depicting these posets as a Hasse diagram, we use the convention 
that a northwest vertex is larger than another in southeast. Here the larger dots and the heavier 
edges indicate the top tree. For later use we denote by P — Pj(a) the Shifted shape associated 
with a strict partition a. If P = pj(a) is the shifted shape associated with a strict partition a, 



Figure 2: Shifted shapes C2 



then P-partition 
satisfies 



"KiA < 7T 



(1.7) 
(1.8) 



_7Tn _7Ti2 



7T22 



7T33 



7T14 _ 7Tl5 7Tl 6 _7Tl 7 _7Tl 8 
7T25 7T26 



7T24 



7I"34 



7T44 



Figure 3: P-partition for shifted shape (8, 5, 2, 1) 



whenever the both sides defined. For example, Figurc|3]is a P-partition for shifted shape (8, 5, 2, 1). 

In this paper we mainly consider only two classes, i.e., birds ^3 (Figure 0]) and banners ^ 
(Figure [6]). Let a = (011,02) and = (/3i , /3 2 ) be strict partitions such that or > a-i > and 




i 



Figure 4: Birds C3 



Pi > P 2 > 0. Define the bird P = P 3 (o,/3; /) by 

P = P H U P r U P l U P t 

where 

P H = {(!,.?) : -/ + ! <.?<!}, 



Pn = {(i,j) 
P T = {(£,£) 



« < j < cti + i - 1 (i = 1,2) }, 
./ • ' ' ■ J 1 (i = 1, 2) }, 
2 < i < / + 2} 



as a set and we regard it as a poset by defining its order structure (|1.6[) if and only if the both 
of (£1, ji) and (i 2 , J2) are in Ph U Pr U Pl or in Pr (see [§J Table 1 and Figure 5.3]). We call 
Ph the Lead, Pr the tail, Pr (rcsp. Pl) the right (rcsp. left) wing of P. The Hassc diagram 
of a bird is as in Figure |4l Strictly speaking, we have to impose the condition ct\ = 0,2 + 1 and 
Pi = P2 + 1 to let P be slant irreducible, but here we don't need this condition. For example, the 
left-picture in Figure [5] stands for P = p3((4, 3), (4, 2); 2). We have the chain [w, v 2 ] (resp. [w2,w]), 
which is the head (resp. tail) of P. Recall that a P-partition tt satisfies the condition (|1.5p . When 
P = p3(o, /?; /), we associate the quadruple (er, r; p, 9) with 7r, where 



( T 'j')(i,0e-PL' P - {pi)i=o,...,f, 9 - (0i)i= Ot 



with 



P-1+1 



for (£,j) e Pr, 
= tt(1,£) for(l,£)eP H , 



= 7r(£,£) 



for e P L , 
for (£,£) £ P T . 



(1.9) 
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Figure 5: Bird P = P 3 ((4, 3), (3, 2); 2) and banner P = P 6 ((9, 6, 3, 2); 2) 

Hence we use the convention that po = o"n = m and #o = 022 = ^22 ■ We write 7r = (a, t; p, 0) 
hereafter. If P = P 3 ((4, 3), (4, 2); 2) then it is as the left picture of Figure [7j 

Let a = (a.\,a,2,OLz,a,4) be a strict partition such that oi\ > 011 > 0^3 > > 0, and let / > 2 
be a positive integer. Let P be the set P = Ph U P\v U Pt of lattice points in 1? , where 

P H = {(l,j) : -f + 2<j < 1}, 

Pw = {(«,i) : i<j<ai + (i = 1,2,3,4)}, 

P T = {(i,3) : 3<i</ + 2}. 

We regard P as a poset by defining the order relation (|1.6[) if both of and (^2,^2) are in 

Ph U Pw or in Pt, and call it a banner (see [HI Table 1 and Figure 5.6]). The Hasse diagram 
of a bird is as in Figure 2] in general. Strictly speaking again, we have to impose the condition 
oti = ct2 + 1 to let P be slant irreducible, but we don't need this condition here. We call Ph the 
head, Pt the tail, and P\v the wing of P. We use the symbol P = Pe(a; f) to mean the banner 
associated with a strict partition a and a positive integer /. The Hasse diagram of a banner is 
given in Figure [51 For example, the right picture in Figure [S] stands for P = P 6 ((9, 6, 3, 2); 2). If 
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Figure 6: Banners C§ 

P = Pe(a; f) is the banner, we associate a triplet (a;p,0) with a P-partition it, in which each 
component a = (<Ti,j)(i,j)eS(a), P = (Pi)i=i,...,f, 6 = {Qi)i=i,...,f are defined by 

<Ti,j = 7r(*,j) for (hj) 6 P W, n 1fV , 

ft = tt(1, -J + 2) for i = 1,...,/, fl t - = 7r(i + 2, 3) for i = 1, . . . , /. { > 

Hence we have p\ = an and 6\ = 033. Hereafter we write 7r = (cr; p, 6). For example, the right 
picture of Figure [7] is a Pe((9, 6, 3, 2); 2)-partition. 

Let P be a connected d-complete poset and T its top tree. Let C be a set, called a set of colors, 
whose cardinality is the same as T. A coloring of P a coloring map c of P to the set of colors C. 
P is said to be properly colored if the coloring map c satisfies 
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92 0\\ (J\2 



&22 



f 13 



023 



033 



(714 



024 



034 



015 016 017 018 019 

' TT 

025 026 027 
• • 

035 



?2 044 CT45 



Figure 7: P-partitions 

(CI) c{x) 7^ c(y) if x and y arc incomparable, 
(C2) c(x) ^ c(y) if a; covers y. 
It is simply colored if, in addition: 

(C3) whenever an interval [w, v] is a chain, the colors of the elements c(x) in the interval [w, v] are 
distinct. 

If P is a rooted tree, then it is simply colored by the identity map P — > P, i.e. we assign a distinct 
color to each vertex of P. 

Proposition 1.5. ([TDJ Proposition 8.6]) Let P be a connected d-complete poset and T its top 
tree. Let C be a set whose cardinality is the same as T. Then a bijection c : T — > C can be 
uniquely extended to a proper coloring c : P — > C satisfying the following condition: 

(C4) If [w,v] is a <4 -interval then c(w) = c(v). 

Such a map c : P — > I is called a d-complete coloring. 

For example, in the both picture of Figure [5] because [102,^2] (resp. [wi,wi], [w,v]) is a ^5- 
intcrval (rcsp. ^-interval, G^-interval), wi (resp. w\, w) and V2 (resp. v\, v) have the same color. 
In Figure El fi (rcsp. V2) and V3 (resp. W4) have the same color since [173, i>i] (resp. [174,^2] is a 
c?4-interval, however, the v\ and vi have distinct colors since the both are in the top tree. 

Proposition 1.6. (1) If a is a strict partition with lcngth> 2, then the top tree of the shifted 
shape P = p2(a) is given by 

T = {(!,.?) : 1< j<ai}U{(2,2)}, (1.11) 

and a d-complete coloring c : P — > {0, 0', 1, 2, . . . , ai — 1} is given by 

if i < j, 

if i = j and i is odd, (1-12) 
0' if i — j and i is even. 

Hence we see that P has the top tree Y(l;l,a\ — 1). 

(2) If a and f3 are strict partitions with length= 2 and / > 1 then the top tree of the bird 
P = P 3 (a,P; /) is given by 

T = {(!,]) : -/ + l<j<ai}U{(i,l) : 1 < i < Pi } , 

and a d-complete coloring c : P — > {— /, . . . , — 1, 0, 1, 2, . . . , ai — 1} U {1', 2', . . . , (/?i 
given by 

' j-i if i < j, i.e., (i, j) £ P R , 

if 1 < 3 < i, i-e., 
j - 1 if i = 1 and j < 1, i.e., E P H , 
^-z + 2 if * = i > 2, i.e., (i,j)£P T . 
Hence we see that P has the top tree Y{f;a\ — 1, f3\ — 1). 



c(i,j) 



(1.13) 
1)'} ^ 

(1.14) 
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(3) If a is a strict partitions with lcngth= 4 and / > 2 then the top tree of the banner P = 
Ps(a; /) is given by 

T={(l,j) : -/ + 2<i< ai }U{(2,2)}, (1.15) 
and a d-complete coloring c : P — > {— / + 1, . . . , — 1, 0, 1, 2, . . . , ot\ — 1} U {0'} is given by 

{3 - i if i ± J, 
if i = j and i is odd, (1-16) 
0' if i = j and i is even. 

Hence we see that P has the top tree Y(f; ct\ — 1, 1). 

Proof. (1) is obtained in [H Example 4.3(b)]. (2) and (3) are also obtained from (C1)-(C4). □ 

Let P be a connected d-complete poset and c : P — > C a d-complete coloring. Let Zi (i £ C) be 
indeterminates. For a P-partition tt £ g/(P), we write 



n 



tt(v) 
c(v) 
vEP 



As in [8j p. 412] we associate a monomial z[Hp(v)] to each v £ P, called the hook monomial, which 
is uniquely determined by induction as follows: 

(a) If v is not the top of any dfc-interval, then we define 

z[H P {v)\ = Y[ z c[w) . 

(b) If v is the top of a d^-interval [w, v], then we define 

z[H P (x)} ■ z[H P (y)} 

z[Hp(v)\ = ,„ , „ , 

z[Hp{w)\ 

where x and y are the sides of [w, v]. 

Further we denote z[H p ] = { z[Hp(v)] : v £ P } the set of the hook monomials, and let F (z[H p ]; q, t) 
denote the product of F (z[Hp(v)}; q, t) over v £ P, i.e., 

F(z[H p ];q,t)=l[F(z[Hp(v)};q,t). 

veP 

Let P be a connected ci-complete poset with the maximum element vq, and the rank function 
r : P — > N. Let T be the top tree of P. Take T as a set of colors and let c : P — > T be the 
(i-complete coloring such that c{v) = v for all v £ T. Let P = FU{1] be the extended poset, 
where 1 is the new maximum element of P which covers vq. Then P has its top tree T = T U {1}, 
where c : P -> T with c(l) = 1. 

Definition 1.7. Given a P-partition it £ £?(P), let n : P — > N be the extensions of it defined by 
7i(l) = 0. Define a weight Wp(cx; q, t) by putting 



f(a(x) - a(y); e(x, y))f(a{x) - <r(y); e(x, y) - 1) 



W P (ir;q,t) = — fl^g^gW ; (L17) 



,vep 



where c(x) ~ c(y) means that c(x) and c(y) are adjacent to each other in T, and 

rfa) - r(x-) - 1 r(y) - r(x) 

d(» 3 y) = 2 ' e ^ x ' ^ = 2^ ■ 

Note that if c(x) ~ c(y) then r(y) — r(x) is odd, and if c(x) = c(y) then r(j/) — r(x) is even, hence 
d{x,y) and e(x,y) are nonnegative integers. 
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Now we quote Okada's (q, t)-hook formula conjecture. 

Conjecture 1.8. (Okada [5]) Let P be a connected d-complete posct. Using the notations defined 
above, we have 

Y, W P (ir;q,t)z* = F(z[H p ];q,t). (1.18) 

Okada has proven this conjecture for shapes and shifted shapes. The purpose of this paper is 
to prove his conjecture for birds and banners. 

Theorem 1.9. Okada's (q,i)-hook formula conjecture is true for birds and banners. 

Given a P-partition it G stf(P) for the shifted shape P = P2(a) for a strict partition a, we 
write 

jND /„. .\ TT TT /(^'U ~ Ki-rn ,j-m-l', m ) f {^i ,j ~ 7T»-m- 1, j-m ', m ) n . „s 

fa (n,q,t)= [I 11 f(jT . ._ JT . ; . m)f(jT . ._ w . —y (1.19) 

tT>r ,\ TT TT f( n i,i ~ Ki-m-l.i-m; m)f(-Ki i - 7Tj_ m _2,i- m -i; TO + 1) , . 

J a \' K \q 1 t)= — — — — . (1.20) 

,... A "V- - Ki-m.,i-m, m)j\Fi,i ~ Ki-m-2,i-m-2, m + 1) 

Here we use the convention that nij = if i < or j < 0. Further we use the following short 
notation. Let m and n be positive integers such that m < n. When p = (p m , . . . , p n ) and 
9 = (9 m , ...,9 n ) satisfy 

< Pn < ■ ■ ■ < Pm < 9 m <■••< 6 n , (1.21) 



we write 



® m \PiVi 9)*J = 777 r— t • (1.22) 

Proposition 1.10. (1) Let a be a strict partition of length r and P = P2{a) the associated 
shifted shape. If ir = (7Tij)(ij)ea i s a P-partition (|1.7|) satisfying the condition (|1.8p . then its 
weight Wp(7r; g,i) is given by 

Wp(tt; q, 1) = /°(tt; <?, t) /* d (tt; <j, f). (1.23) 

(2) Let a and j3 be strict partitions of length 2. Let / > be a positive integer, and set 
P = Pj (a, P; f) to be the bird associated with a, (3 and /. If n = (a, r; p, 9) is a P-partition 
satisfying the condition (|1.9[) . then its weight Wp(7r;g, f) is given by 

u/ , ,v /(gag - gigi 0)/(r 22 - r 12 ; 0)/( P/ ; 0)/(fl /; / + 1) 

Wf(7T <?,£) = — T-TJ-, 77 

./(ct 22 - crn;0)/((722 - cm; 1) 

x *l(pAq,t)f™(v-,q,t)ff D (T-,q,t). (1.24) 
Here we use the convention that an = t\\ = po and cr 22 = ^22 = #o- 

(3) Let a be a strict partition of length 4. Let P = Pe(a; /) be the banner associated with a and 
/. If 7r = (a; p, 9) is a P-partition satisfying the condition (|1.10p . then its weight Wp(ir; q, t) 
is given by 

W p (tt; q, t) = f(p f ; 0)f(9 f ; f + l)$f (p, 9; q, i)/£(cr; g, i)/^ D (cr; g, t). (1.25) 

Here we use the convention that an = p\ and (733 = ^i. 

Proof. For (1) f|l .23[) is exactly the same as [H Theorem 1.2 (9)]. For (2) and (3), one can compute 
Wp(7r; q, t) directly from the definition (|1.17|) . □ 
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Proposition 1.11. (1) Let a be a strict partition of length r and P = P^ia) the associated 
shifted shape. Let n be an integer such that n > ot\, and let a c be the strict partition formed 
by the complement of a in [n], i.e., 

...,a r }U {a{, . . . ,a£_ r } = [n]. 

We write yo = Zo> (sec Proposition [O)] (1)) hereafter. Then we have 

F(z[H p ];q,t) = J] F (z^cZ aj ; q.tj J| F (z ai ; q,t)J]_F (w z ai z aj ;q,t) , (1.26) 



where 



w = yo/ z Q and z t = Y\k=q Zk (i = 1, ■ ■ ■ ,n). if r is odd, 
w = zq/uq and = y 111=1 z fc (« = 1,..., n). if r is even. 



(2) Let a = (0:1,(22) and j3 = (fix, fo) be strict partitions of length 2. Let / > be a positive 
integer, and set P = J^a, /?;/) the bird associated with /, a and /3. Let m, n be integers 
such that m > £(a) and n > and let a c (resp. (3 C ) be the strict partition formed by 

the complement of a (resp. j3) in [m] (resp. [n]). We write ?/i = Zi> for £ = 1, . . . , y3i — 1 and 
x% = Z—i for i — 1, . . . , /. Further we may write xq = yo = zq. (See Proposition 11.61 (2)). 
Then we have 

/ 

F(z[H p ];q,t) = J] F (z^z aj ; g.t) [J F «•*) II F (**5 ?•*) 

/ I ~2 2 



X 



a;.. 



where = Xi for £ = 0, . . . , /, & = nJUi for £ = 1, ... , n, and Zj = Y\l=i z k f° r 

£ = 1, . . . ,m. 

(3) Let a = (ot\, a% 03, 04) be a strict partition of length 4. Let _P = _Pg(/;a) the banner 
associated with a and (3. Let n be an integer such that n > 4 = ^(a), and let a c be the 
strict partition formed by the complement of a in [n\. We write yo = zo> and Xi = z_.; + i for 
£ = 2, . . . , / (see Proposition 11.61 (3)). Hereafter we may use the convention that x\ = zq. 
Then we have 

/ / /~ 2 4 \ 

F(z[H p ];q,t) = J] F (zg}z aj ; g.tj [] F g.t) [] F [% w 2 [] z Qi ; q.t\ 

a?<aj i=2 i=2 \ 1 i=l ) 

4 

xY[F(z ai ;q,t) F (x 2 wz ai z aj ;q,t) , (1.28) 

i=l l<«<i<4 

where ty = ^, Xi = ]\{=t x i for i = 1, . . . , / and z, = y 111=1 z fc for £ = 1, . . . , n. 
Proof. (1) If P = P2(a), then we have 

{Za(Za j+1 iii<j<r, 

z ai i£i<j = r, 

z~l z a% iii<r<j, 
«i — j+i 

as stated in (81 §3, Proof Theorem 1.21. 
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(2) If P = P 3 (a, (3; f), then we have 



(3) If P = P e (a; /), then we have 



( 7 2 


Y\k,i= 


X -3 + l 








j + l 






Xi-2 





VkZi 



z[H P {i,3)\ 



x -j + 2 Vo 



X t -2 



if i = 1 and -/ + 1 < j < 0, 

if l<i,j < 2, 
if 1 < i < 2 < j, 

if 1 < i < 2 < i, 

if 3 < i = j < / + 2. 



if i = 1 and -/ + 2 < j < 0, 

if 1 < i < j < 4, 

if 1 < i < j = 4, 
if 1 < i < 4 < j, 

if 3 < i < f + 2 and j = 3. 



□ 



2 Macdonald polynomials 

In this section we recall the fundamental properties of Macdonald 'polynomials and consider it's 
application. Especially Theorem 12.21 and it's corollary will play an important role in the next 
section. 

We follow the notation and terminology of [7] for the symmetric functions. If A and fi are 
partitions then fj, C A if jj, is contained in A, i.e., /i.; < A; for all % > 1. If ll C A then the 
skew-diagram X/ ll denotes the set-theoretic difference between A and /i, i.e., those squares of A 
not contained in ll. The skew diagram X/ ll is a vertical r-strip if |A — fi\ = |A| — |yu| = r and if, 
for all i > 1, Ai > /ii is at most one, i.e., each row of A — \x contains at most one square. The 
set of all vertical r-strips is denoted by ^ and the set of all vertical strips by V = l+)^_ The 
skew diagram Xj ll is a horizontal r-strip if ]A — juj = t and if, for all i > 1, A^ — it\ is at most 
one, i.e., each column of A — fi contains at most one square. For two partitions A and fi, we write 
A >- ll if A D ii and X/jj, is a horizontal strip. Note that X/ ll is a horizontal strip if and only if 
Ai > fJ,i > A2 > fJ,2 > ■ ■ ■ ■ The set of all horizontal r-strips is denoted by J%? r and the set of all 
horizontal strips by Jf. Let s = be a square in the diagram of A, and let a(s) and l(s) be 
the arm- length and leg- length of s, given by 



a(s) = A, - j, 
Then wc define the rational functions let 

b\{s) = b\{s;q,t) 
and (3.6)] [3 VI.7 (6.19), VI.7 Ex.4] 



l(s) = A' - i 



l-q a 



l_ga(»)+l t l(a) ' 
E 



if s e X, 
otherwise, 



bx(q,t) :=l[b x (s;q,t)=l 



i>l m>0 



fq,t(Xi — Aj+ m +i; m) 
fq,t(Xi — Aj+ m ;m) 



b x (Q,t)-= 11 b x (s;q,t)=[[ [[ (A ._ A , m) ■ 

oven - m even 

sEA 
a(») odd 



(2.1) 
(2.2) 
(2.3) 
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If x = (x±, X2, ■ ■ ■ ) and y = (y±, j/2, • • • ) are two sequences of independent indctcrminates, then we 
write 

U{x; y- q,t) = U = JJ F{x m q, t). (2.4) 

Let ©„ denote the symmetric group, acting on x — [xi, . . . , x n ) by permuting the Xi, and 
let A„ = Z[xi, . . . ,a; n ] " and A denote the ring of symmetric polynomials in n independent 
variables and the ring of symmetric polynomials in countably many variables, respectively. For 
A = (Ai, . . . , A n ) a partition of at most n parts the monomial symmetric function m\ is defined as 

m\(x) = ^ x a 

a 

where the sum is over all distinct permutations a of A, and x = (x\, . . . , x n ). For £(X) > n we 
set m\(x) = 0. The monomial symmetric functions m x (x) for £(X) < n form a Z-basis of A,,,. 
For r a nonnegative integer the power sums p r are given by po = 1 and p r = m< r ) for r > 1. 
More generally the power-sum products are defined as p\ (x) = px 1 (x)p\ 2 (x) ■ ■ ■ for an arbitrary 
partition A = (Ai, A2, . . . ). Define the Macdonald scalar product (■, -) q _ t on the ring of symmetric 
functions by 

" 1 - q x < 

(p\,Pn,) q ,t = 6\pz x n n 1 ^ 

i i—1 

with z\ = Y[i>i^ mim ^- an d m i — Trii(X). If we denote the ring of symmetric functions in A„ 
variables over the field F = Q(q,t) of rational functions in q and t by A n! f, then the Macdonald 
polynomial P\{x) — P\(x\ q, t) is the unique symmetric polynomial in A n> F such that [VI (4.7)]Mac: 

-Pa = ^ u Xll (q,t)m^(x) 
ij,<\ 

with u\\ = 1 and 

(P\,P»} q ,t = ifA//i. 

The Macdonald polynomials P\(x;q,t) with £{X) < n form an F-basis of A n $- If ^(A) > n 
then P\(x;q,t) = 0. P\(x;q, t) is called Macdonald 's P-function. Since P\(xi, . . . , x n , 0; q, t) = 
P\ {xi , ■ • ■ , x n ; q, t) one can extend the Macdonald polynomials to symmetric functions containing 
an infinite number of independent variables x — (x±, x%, . . . ), to obtain a basis of F = A ® F. A 
second Macdonald symmetric function, called Macdonald's Q-function, is defined as 

Qx{x; q, t) = b x (q, t)P x {x; q, t). (2.5) 

The normalization of the Macdonald inner product is then (P\, Qfi)q,t = ^a^ for all A, p, which is 
equivalent to 

]T Px(x; q, t)Qx(y; q, t) = U(x; y; q, t). (2.6) 

A 

(See VI.4, (4.13)].) Let g r {x;q,t) := Q {r) {x; q,t), or equivalently, [3 VI.2, (2.8)] 

OO / \ OO 

( X . v .a) =2^9r(x;q,t)y . 

Then the Pieri coefficients 4>x/n and tpx/n are given by [7j VI. 6, (6.24)] 

P^(x;q,t)g r (x;q,t) = ^ 4> x/ll (q,t)Px(x;q,t), 

A 

Q^(x;q,t)g r (x;q,t) = ip x/lJ/ (q,t)Qx(x;q,t). 

X 

X—fieJ&r 
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Another direct expressions for 4>\/^ and ipx/n is given in [JJ VI. 6, Ex.2] as 

, , a tt /(A, - Hj-J - i)f(m - Aj+i; j - j) 

Here we use these expressions to rewrite Okada's (q, i)-hook formula conjectures by the Pieri 
coefficients. For any three partitions A, fi, v let f* v be the coefficient P\ in the product Pf_ L P v : [7J 
VI (7.1')]: 

P„(x;q,t)P ll (x;q,t)=J2f^P x (x;q,t) (2.9) 

A 

Now let A, \i be partitions and define Qx/n G by 

Qx/fj,{ x j q,t) = Yl &Q»te *)• ( 2 - 10 ) 

Then Qx/^x; q, t) = unless X D fi, and Qa/^ is homogeneous of degree |A| — which is called 
Macdonald's skew Q-function. We define Macdonald's skew P '-function Px/n as 

Qx/^x; q, t) = ^Mp A/A1 (x; q, t). (2.11) 

holds. Let T be a tableau of shape A — /i and weight i/, thought as a sequence of partitions 
(A(°),...,AM) such that 

/i = A<°> c A« c • • • c A« = A 
and such that each A^ — A'* -1 ' is a horizontal strip. Let 

r 

i=l 
r 

ih(q,t) = Y[iP\w/\u-i)(q,t)' 

i=l 

Then wc have VI, (7.13), (7.13')] 

T 



T 

X , 



summed over tableaux T of shape A — fj,, where x T = 111=1 X \ X A '■ It a l so holds [JJ VI. 7, 
(7.9) (7.9')] 

Qa(z, z; q,t)=^2 Q\/ti( x ' z '> ?> t)Qn( x , z \ Q, *)> ( 2 - 12 ) 

Px{x,z;q,t) = ^Px/^{x,z\q,t)P^{x,Z)q,t), (2.13) 

where the sums on the right are over partitions /icA. The following lemma has appeared in the 
proof of [131 Proposition 2.2] (also see [7J 1.5, Ex.26] and [14[ Proposition 5.1]). 

Lemma 2.1. Let and v be partitions, and x = (a?i, £2, ■ • ■ ) and y = (yi, 1/2, . ■ ■ ) arc independent 
indctcrminatcs . 

^Qx/^x;q,t)Px/ v {y]q,t) = U.{x;y;q,t)'Y^Q v / T {x)q,t)P ll / T {y;q,t) (2.14) 
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Proof. First if wc use (|2~T2)) ([2J3|) and ((2^6)) . then we have 
E E Qa/m^A/^W^H 

p,,v A 

= E E ( 3 a ( x ' 

= n(x; y)II(x; ty)n(z; y)II(z; iu) 

= n(x; y) E Q&)Pt(w) E Q-jCO^fo) E Qt(«)-Pt(^) 

f 77 T 



n(x; y ) E QfW^E^/^WE^H 



by (HU) and d23 



by and flUT) 



n(x;?/) E Qv/T( x ) p n/ T (y)QtJ.(z)p v {w) 



Hence, by comparing the coefficients of Qp,(z)P u (w) in the both sides, we obtain the desired 
identity. This completes the proof. □ 

In [13] Vuletic has presented so-called a generalized MacMahon's formula. The following theorem 
gives a generalized form of [T3J Proposition 2.2], which we use in the proof of Okada's conjecture. 

Theorem 2.2. Fix a positive integer T and two partitions fjP and fi T . Let a; , . . . , x T_1 , y 1 , . . . , y T 
be sets of variables. Then we have 

T 

E II Qavm*- 1 «> *) p avm« (»*;«. *) 

(Ai, M i,A 2 ,...,A T )i=l 

= [] n(x^y J '; 9j t)^Q Mr/v (s ,...,x r - 1 ;?,t)P^o /l/ (y 1 ,...,y r ; ? ,t) (2.15) 

0<i<i<T v 

where the sum runs over (2T — l)-tuples (A 1 , fj, 1 , A 2 , . . . , /i T_1 , A T ) of partitions satisfying 

At C A 1 D C A 2 3 /i 2 C ■ • • D [i T - x C A T D fj, T . (2.16) 

Proof. Our proof is similar to that of [131 Proposition 2.2]. We proceed by induction on T. If 
T = 1 then Lemma T2. II is nothing but the desired identity. Assume T > 1 and (12.151) holds up to 
T — 1 . We need consider the sum 

T 

(A 1 ,/i 1 ,A2,...,A' r )i=l 

where the sum runs over (A 1 , fi 1 , A 2 , . . . , A T ) satisfying (|2.16[) . First fix (fi 1 , . . . , ^ T ~ 1 ) and take 
the sum over (A 1 , . . . , A T ) using (|2.14j) . Then we obtain 

T T 

1=1 (TKnK...,r T )i=i 
where the sum runs over (A 1 , fi 1 , A 2 , . . . , A T ) satisfying 

fi v DrC(i 3r C/i D '"C|i D r Cfi . 
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By the induction hypothesis we can suppose 

T 

e En^v^ -1 )^- 1 /-^) 

<ji l ,T 2 ,...,fi T - 1 ) i=l 

= [] n(x l ;y J+1 )Y,QTT/Ax ,---,x T - 2 )Pry»(y 2 ,-.-,y T )- 

0<i<j<T-l v 

Hence, substituting this identity into the above S, we obtain 
S = J] n(x*;y->) Q^ /TT (x T - 1 )P^ /T i(y 1 )Q T T /v (x ,...,x T - 2 )P T i /l/ (y 2 ,...,y T ), 

0<i<j<T (t 1 ,u,t t ) 

where the sum runs over (r 1 , v, t t ) such that 

fi° D t 1 D v C t t C fi T . 

Applying (|2.12j) and (|2.13[) . we obtain the desired identity for T. This completes the theorem. □ 

We define P& Jx;q,t) and Qj? A , (x; q, t) for a pair (A, fx) of partitions, a set x = [x\, x%, . . . ) of 
independent variables and 6 = ±1 by 

p5 fP A / M 0;g,*) if<y = +l, 4 , . _ fQ\/n(x;q,t) if 5 = +1, 

[Q M/A (x;g,t) if d = — 1, [-P m /a(^;'7^) ifd = -l. 

Here we assume A D fi if 5 — +1, and Ac^if<5 = — 1. 

Corollary 2.3. Let n be a positive integer, and e = (ei, . . . , e„) a sequence of ±1, Fix a positive 
integer T and two partitions A and A™. Let x 1 , . . . , x n be sets of variables. Then we have 

n 

e n^-*]^.*) 

(ASA 2 ,...^"- 1 ) i=l 

JJ n( 2; l ; ^ '; g, i) £ QA-y^Kle^-i; 9, t)PAO/,({^} £j=+1 ; q, t), (2.17) 

(e i ,z j ) = (-l, + l) 
71 

e n^-w^;**) 

(A 1 ,A 2 ,...,A" _1 ) 4=1 

[J nix- 4 ; aJ; q, t) £ P A n /l/ ({x i } ei =- 1 ; «, t)Q X o /v ({x%^ +1 ; q, t), (2.18) 

( ei , ej ) = (-l, + l) 

where the sum runs over (n — l)-tuples (A 1 , A 2 , . . . , A™ -1 ) of partitions satisfying 

j X 1 - 1 D X 1 ife l = +l, 
| A'- 1 C A ?; if £, = -!. 



(2.19) 



Proof. Taking T = n in Theorem 12.21 we have 

n 

E II Oava«-i (x*- 1 ;!, t)PA*/» <X*; q, t) 

(A 1 ,A 1 ,A a ,...,A n ) «=1 

= [] n(x i ;P; 9 ,t)2g A , / ,(x 1 ...,^^ J i)j\. / „(y 1 ,... ) r; !l t) 

0<i<j<n v 

where the sum runs over 

A c A 1 d A 1 c A 2 d A 2 c • • • d A"" 1 cA"d A". 
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Put X 1 - 1 = and Y l = x l if = +1, and X l ~ Y = x l and Y l = if = -1. Since P A/ v(0; q, t) = 

Q\/„(0;Q,t) = I ^ ^' we obtain A 4 = < * ' Hence we have the condition 

A/M ; |0 otherwise, (A 1 if e* = — 1. 

(1 2 . 1 9 [) for the sum. Since 



Q A , /A ,- 1 (X*- 1 ;q,t)P A . /A >(Y l ] q,t) 
the left-hand side equals 



P x >-i/\*(x l ;q,t) if C£ = +1, 
QxVA*- 1 ^ ;<?>*) ife» = -l. 



E Il'fr-W* 1 ;**)- 

(A 1 , A 2 ,..., A™ -1 ) i=l 

Meanwhile the right-hand side becomes 

J| n(a; 1 ; a; J '; q, t) ^ Qav^KK^-i; 9, t)P X o /u ({x% i = +1 ; q, t). 

(e it e 3 = (-l,+l) 

This proves f|2.17[) . The other identity can be proven similarly. □ 
Theorem 2.4. (Warnaar [13 Proposition 1.3, (1.17)]) 

s «. *> - n (1 + T;'y ; ' 2) ~ n <**» 

A i>:L ! y yoo ^ . »,j-i^j,yyoo 

where r(A) is the number of rows of odd length. 

Applying [3 VI.2, (2.14)] to the both sides of ([2~2"0|) . we obtain 

Corollary 2.5. 

x; ^'^(g, t)JM«; ?, *) = n ^rr^ n f^f- . ( 2 - 21 ) 

A 4>1 \ wx i.,q)co { . (XiXj^qjcc 

Proof. First, if we take logarithm of the right-hand side of (|2.20p . then we have 

log n {t l XiXf : q } o ° =EE o°g (! - ? rte **i) - lo § c 1 - 1****)} 

^ n 1 - n" ^ J 

n>l * »<j 

^ ^ 1 — t™ 

= 2 E - • — -Jfcnfr)} • 
n>l q 

Applying the F-algebra homomorphism w q j to this formula, and using w q jp r {x) = (~ l) r ~ 1 \Z. 9 t r Pr{x) 
(see VI.2, (2.14)]), we obtain 

i<j J n>l 

Similarly, since 



TT (.qtxj-q 2 )^ ^1 1 - g"f" 
108 n = E - ■ 7E^n(z), 



i>l w > i ra>1 
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we have 

W «>* Ml ( X 2. a 2) = ~ ~ ■ !_ f 2n E 
i>l ^'' 7 >°° n>l 71 



Finally, from 



we obtain 



Hence we obtain 



n>l 



( 1 "l™ ?/> rl 

log [J (1 + = — 



n (! + = e e - • t^f w "^- 

i i n> 1 

(1 + wxi) (qtx^q 2 )^ f-r {tXiXj\q) j-r (qwx^t)^ yj {qXiXj]t) 



Now, applying 

%,t-PA (x\ <?, *) = Qa' (a;; t, q) 
w q , t Q\ {x; q, t) = P x > (x; t, q) 

(0 VI. 5, (5.1)]) to the left-hand side of (|2.20[) . and swapping q and t, we obtain the desired formula 
(IOTP . □ 



From (|2.21[) . we easily obtain 

h ci (n +\ P, f„-n A — TT i 

(wx^^oo 11 (wXiXjiq)^ ' 



* b x (q,t)P x (x;q,t)=[[ [[ , (2.22) 



A 

and 



-^v^, w; g, t) = n n (2-23) 

A t>i (-^'^oo fjr {wXiXj^q)^ 

3 (g, £)-hook formula and Macdonald polynomials 

In this section we rewrite the left-hand side and the right-hand side of Okada's conjecture using 
the Macdonald polynomials. In Proposition 1 1 . 101 we give the left-hand sides for birds and banners, 
and in Proposition 1 1 . 1 ll we give the right-hand sides. We rewrite these formula into Theorem 13.21 
and Theorem 13.31 We will sec Corollary 12.31 and Corollary 12.51 plays a key role in the proof. 



We define </>^ (q, t) and Tp? x (q, t) for a pair (A, /u) of partitions and 6 = ±1 by 

,6 , .x j$\/»(q,t) if<5 = +l, s (ip\/n(q,t) i£S = +l, 



Here we assume A y fj, if 6 = + 1 , and A -< \i if 6 = — 1 . We also write 

1^ - iA& = 



|A-/i| if<5 = +l, 

|/i-A| if<y = -i. 



Let n be a positive integer. Let e = (ei, . . . , e n ) be a sequence of ±1. Let (A , A 1 , . . . , A") be an 
(n + l)-tuple of partitions such that A 1 " 1 >~ A 4 if e = +1, and A* -1 -< X 1 if e = —1. Then we write 

n n 

0[A°,Ai,...,A"]M) = II^V- 1 ^] ^A»,A 1 ,... 1 A»](8>*) = n^A*-i,A*]fe*)- 

i=l i=l 
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Let a be a strict partition, and let n be an integer such that n > ot\. Define a sequence 
e = e„(a) = (ei, . . . , e„) of ±1 by putting 



+ 1 if k is a part of a, 
— 1 if fc is not a part of a. 



For example, if a = (8,5,2,1) and n = 10, then we have e = (+ H 1 1 ). Let 

7r G £?{P) a P-partition for the the shifted shape P = p2(a). For each integer k = 0,...,n 
we define the fcth trace 7r[fc] to be the sequence (. . . , i^2,k+2, ^l.fc+i) obtained by reading the fcth 
diagonal from SE to NW. Here we use the convention that ir[k] = if k > a\. For example, if tt is 
the P-partition of shifted shape a = (8, 5, 2, 1) in FigurcEl then we have tt[0] = (7^4, 7733, 7T22, ttii), 
t[1] = (7r 3 4,7r 2 3,7ri2), 7r[2] = (^4,^13), 7r[3] = (^25,^14), 7r[4] = (7126,7115), 7r[5] = (tti 6 ), tt[6] = 
(7r 17 ), tt[7] = (tti 8 ), tt[8] = tt[9] = tt[10] = 0, and 

?r[0] y tt[1] >- tt[2] -< tt[3] -< tt[4] >- tt[5] -< tt[6] -< tt[7] y tt[8] -< tt[9] -< tt[10]. 

By direct computation one can easily check 

W P (n;q,t) = 6^[o](ff > *)^fip] i .„ )W[ i ]](9>*) = ^[oi^Ioi/irW^W/irpl^plAp] 

x 07r [4] /tt [3] Vv [4] /tt [5] [6] /tt [5] 0tt [7] /tt [6] [7] /ir [8] 0ir [9] /ir [8] 0w[lO] /tt [9] • 

In the following we write 

/(/> m ;O)(0 m ;m+ 1) 



*£(x;p,0;?,f) = $£(p,0;g,t) 5? 



■Pi+Bi-Pi- 

h 

i—m+l 



in short, where p = (p m , ■ ■ ■ , pn) an d $ = (#m, • ■ • , $n) satisfy (|1.21[) . and x — (x m , . . . , x n ) are 
indctcrminates. For example, if % = (cr, r; /) is the P-partition of the bird P = P^{a,P;f) for 
a = (4, 3), /3 = (4, 2) and / = 2 (see Figure [7]) and satisfies (|1.9[) . then we have 

Wp(tt; 9 , t) = $g(p, 0- q } t)tfW^ m (q, t)^S,..., r[4]] (?, *)■ 

Further, if 7r = (cr; /) is the P-partition of the banner P = Ps(a; /) for a = (10, 6, 3, 2) and / = 2 
(see Figure [7]) and satisfies (jl.lOp . then we have 

^ P (7r;q,t) = *ftp, 6; q, t)bf [0] (q, i)^,...,^]] («>*)■ 

Proposition 3.1. (1) Let P = P 2 (a) be the shifted shape associated with a strict partition a 
such that £(a) = r, and let n be an integer such that n > a-y. If it G s&{P) is a P-partition 
satisfying the condition (|1.8I) . then we have 

W P («;q,t) = 6? m («,t)<g,.., ir[B]] (g l t) - ^|^5S....ir[-]](«' t ) M 

and 

T [0]|-7-(ir[0]') -pj_ ei ( Q ) |7T [» — 1] — 7T [t] | e< (cj 



( = 1 



jj-^ww-q-^kw (32) 



where w and ii (1 < i < n) are as in Proposition II . 1 ll (1). 

(2) Let a = (ai,a<2) and j3 = (fii,^) be strict partitions such that £{a) = £((3) = 2. Let / > 
be a positive integer, and set P = Pa(a, /3; /) the bird associated with a, /3 and /. Let m 
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(resp. n) be a positive integer such that m > ati (rcsp. n > If tt = (a,r;p, ff) is a 

P-partition satisfying the condition (|1.9[) . then we have 



^(7r;g J t) = ^(p J fl;g,0<S,..., ff M](9. f )4M....,r[n] 



(3.3) 



and 



/ 

i(/3) |T[j-l]-T[i]| ei(y3 ) tt ^i+ei-pi-i-ej-i 



4 (a) k[i-l]-o-[i]| ei( „) 
i— 1 z— 1 z— 1 



where Xj (0 < z < /), yi (1 < z < n) and 2i (1 < i < m) are as in Proposition !!.!!! (2). 

(3) Let a = (ai, 0% 0^3, «4) be a strict partition such that Oct) = 4. Let P = Pe(a; /) the banner 
associated with a and /. If tt — (a; p, 9) is a P-partition satisfying the condition (jl.lOp . then 
we have 



W p {tt; q, t) = 0; q, t) b%\ 0] (q, t) ^fo],...,o>]] (?, *) 



e(a) 



and 



7/ 



i=2 i=l 



(3.5) 
(3.6) 



where u>, Xj (1 < i < /), and Zi (1 < z < n) are as in Proposition 1 1 . 1 il (3). 
Proof. (1) From (fTTT^|) and we have 



'TT /(7r[l],-7r[0] j + 1 ;j-») T-rn ,e<(a) / ,x / v 

Hl<j<J /Ml]*-wM,-;j-i) Hi=2 rr OT [i_i] )OT [i]]W)^ 11 e H a J 

j-r fO#^jiH tt" foil ifeiff*) 

Similarly, from (|1.20j) and (f!T2"j) we have 
Hence we obtain (|3.1[) from f|l .23[) since 



itg^^^t) if Cl(a) 



ei(a) 

7r[0],7r[l]] 



(9,0 



.ni<i<j to 



ni<i<j to 



i\3— 0/O 



if ei(a) 



Meanwhile, (|3.2[) can be easily obtained from 



r.^r— 1.1 — l+7Ti — 3,r— 3 



+ n 



1 7T [i — 1] | — | 7T [*] 



(2) As in (1) we have 



f/(a 12 -<T U ;0) n2=a^[^i] I<T[ i|](«,*) 



\IiS2. 



fi^z—^vi',^) 

From (fTTT^) and f[^7|) we have 



— q , 22;Q)/(q , 23-o , ii;i)/(gi2-q'ii;0) tt" J, e '( a ) 



if ei(a) = +, 



n<=2^[a[i-i] lff [f]](9i*) if£ i(a) 



7(na-r li; 0) nr=2^;[f2i ])T[ i]i(9.*) if ^08) = +, 

/ (r23 " T22;0) ;a3^::;oi /(Tl2 - Tli:0) n^SJW**) if ^) = +■ 
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Hence, if we use (|2~7|) or (|2.8p . then we obtain ((331) from (|1 .24|) . On the other hand, ([3^4)1 is 
easily obtained from 

f m n 

i—1 i—1 i—1 

using Zq 11+<722 ~[\{ =0 x? i+9i = (z xi) Po+e ° nf=i x i Pz+S '~ p ' - 1 ~ 6 '- 1 ^ where we use the conven- 
tion (Til = PO an( l 0"22 = #o- 

(3) As in (1) we have 

f»{a;q,t)f^{o;q,t) = 6Sf [0] (?,*)<g,... l(r[B]] (?,*)- 
Hence we can obtain (|3.5[) . Meanwhile, (|3.6[) can be obtained from 

i=2 V^O/ i=1 

using n{=2 x i i+ei = 2?2 1+61 Tlf=2 ^r +9, Pl 1 9,-1 1 where we use the convention pi = <rn 
and #1 = ^33. 

□ 

Theorem 3.2. (1) Let P = P^ia) be the shifted shape associated with a strict partition a of 
length r. Let n be an integer such that n > a±, and let a c be the strict partition formed by 
the complement of a in [n\. Then we have 

W P (ir,q,t)z*= II F(^z ai )Y, w ^^^,t)I\(za 1 .-.,z ar iq,t), (3.7) 

where w and % (i = 1, . . . ,n) are as in Proposition II . 1 ll (1). 

(2) Let a = (0:1,0:2) and /3 = (/3i, /3 2 ) be strict partitions such that 1(a) = l(/3) = 2. Let / > 
be a positive integer, and set P = Py,(a, (3; /) to be the bird associated with a, (3 and /. Let 
m (resp. n) be a positive integer such that m > cti (resp. n > /3i). If ir = (a,T\p,8) is a 
P-partition satisfying the condition (|1.9[) . then we have 

x X] ^o(^P> e ;?'*)^(eo,Po)(^o^ai,Soi a2 ;g,t)Q(e , M )(y l a 1 ,z / j 2 ;g,t). (3.8) 

where the sum on the right-hand side is taken over all pairs (p, 9) with p — (po, . . . , pf) and 
6 = (0 o ,...,6f) satisfying 

< p f < ■ ■ ■ < po < 9 Q < ■ ■ ■ < 6 f . (3.9) 
Here Xi (0 < i < f), yi (1 < i < n) and Zi (1 < i < m) are as in Proposition II. Ill (2). 

(3) Let a = (ai, «2, «3 7 04) be a strict partition such that £(a) = 4. Let P = Pg(a;/) be the 
banner associated with a and /. If n = (a;p,6) is a P-partition satisfying the condition 
(|1.10p . then we have 

W P (ir,q,t)= II F(z^}z Xl ) £ ^{(x; p,9;q,t) 

X (x 2 w) A2+A4 6^(g,t)PA(? Ql ,^a 2 ,2'Q3 5 2 Q 4;9,0 1 ( 3 - 10 ) 
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where the sum on the right-hand side is taken over all triplets (X,p,0) with A = (Ai, A2, A3, A4), 
p = (p!,...,pf) and 6=(6i,..., f ) satisfying 

A4<A 3 <A 2 <Ai, < p f < ••• < pi = A 4 , A 2 = 0i < ••• < 6 S . (3.11) 

Here w,Xi (1 < i < /) and zi (1 < i < n) are as in Proposition II .111 (3). 

Proof. (1) Since 

i>Tr[i-l]/ir[i]{<l,t)z i ^ [l ~~ 1] ~~ 7rll]l = P^[i-l]/ w \i](Zi;q,t), 
<Pir[i]/ir[i-l](<l,t)z i ~^ l *~ 1] ~ 7Tll]l = Q^}/^-!]^' 1 ^^) 

(see [3 VI.7, (7.14)(7.14')]), we can use (j2"T7)) to take the sum of the product of (JSH) and 
(13.21). then we obtain 



7T 

= J] F E ^[0] (9. t)w^-^» 2 P n[0] (z ai ,..., z ar ;q,t), 

al< ai tt[0] 

where the sum on the right-hand side runs over all partitions 7r[0]. 

(2) Again, using (|2.17[) to take the sum of the product of (|3.3p and (|3.4|) . we obtain 

x E 5o(P^)^o 0+e ° p ^[o](^i^a2;9^)Qr[o](^ 1 ,^2;9^) I 
0,0) 

where the sum on the right-hand side runs over all pairs (p,#) satisfying (|3.9[) with <r[0] = 
t[0] = (9q, po). Finally we use x p G a+6a P {Bo ^ Q ) (z ai , z Q2 ; q, t) = P^ , po) (x z ai ,x z a2 ;q,t). 

(3) Using (|2.17|) to take the sum of the product of (|3.5|) and (|3.6p . we obtain 

J2w P (TT;q,t)z* = J] F(2^2 Ql )5^*{((7[0],p,e) 

x (^r^+^^o] (g, ^H !^^'))/^ (z Ql , z Q2 , z Q3 , F a4 ; g, t), 
where the sum on the right-hand side runs over all triplets (<j[0],p,0) satisfying (|3.11|) . 

□ 

If we apply Warner's formula (|2.23p to (|3.7p we can obtain the (g, i)-hook formula (|1.26p for shifted 
shapes. This gives another proof of [HI Proposition 4.5 (b)]. Now we look at the right-hand side of 
the conjectured identies in the cases of Birds and Banners. From Proposition II . 1 ll we can derive 
the following theorem. 

Theorem 3.3. (1) Let a = (0:1,0:2) and (3 = (/?i,/3 2 ) be strict partitions of length 2. Let / > 
be a positive integer, and set P = Ps(a, (3; /) the bird associated with /, a and (3. Let m, n 
be integers such that m > £(a) and n > £((3), and let a c (resp. (3 C ) be the strict partition 
formed by the complement of a (resp. (3) in [m] (resp. [n]). Then we have 

F(z[H p ];q,t) = ]J F (z~}z a ^ J] F 

x E E E E I[m,o)f(k,o)x^- i < 

>■ 1=0 fei kr >0 h i/>o i=l 

<W<» ' i 1 + ...+ tf= i 

x & V 7~ T\ 1 ' p A(Si? ai , ^i^a 2 ; g, t)Qx(y^ , vp 2 ; g, *) (3.12) 
o\\q,t) 

where (1 < i < /, j/i (1 < i < n) and Sj (1 < i < m) are as in Proposition !!.!!! (2). 
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(2) Let a = (c*i, 012, OJ3, a.4) be a strict partition of length 4. Let P ~ Ps(f;a) the Banner 
associated with a and /3. Let n be an integer such that n > 4 = £(a), and let cf be the 
strict partition formed by the complement of a in [n]. We write yo = zo' an d = Z—i for 
i = 1, . . . , /. Then we have 

F(z[H p };q,t)= ]J F (z~}z aj ) 

x E E E E ri/^so)/^^)^-'- 

* 1=0 fe,..,if>0 l2.. -.'f>° i=2 
' : a +...+i /= i 

t) (3.13) 

where w, x% (2 < i < f) and 2j (1 < i < n) are as in Proposition II .111 (3). 

Proof. (1) From (|2.6[) we have 

2 

H F (»iJ//3^ Qi ) = ^Pp(xiZ ai ,xiZ a3 )Q IJt (yp 1 ,yp a ). 

i,j=l n 

By the binomial theorem we have 
/ / 

E F (^)= E II z^ )^' 

i=l fci,...,fc / >0i=l 
f I 2 \ j / 2 \ 2i+-+Jf 

i=i y 1 /u=i / !i,...,! / >oi=i y k,t=l 

By [3 VI.4, (4.17)] and we obtain 

V,), 

From (|1.27[) we obtain 

F(«[F p ]; 9 ,t)= II F(z^z aj ) II F(g^y ft 

/ 

x E E E E n^>°)/(« 

bjj&t) p , ^ [xiz ai , JiZ Q2 ; q, t)Q +l . r i {y Pl , y h ;q,t). 
0u+M 2 (3)*) 



: 



This immediately implies (|3.12j) . 
(2) From Warner's formula (|2.23[) . we have 

4 

F (wx 2 z ai z aj ;q,t) 

2 = 1 l<i<j<4 

^ ^ 4 bp (q, t)P[ 1 (z ai , z a2 , z a3 , z ai , q, t). 
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By the binomial theorem we have 
/ / 

i=2 fe 2 ,...,fc/ >0-i=2 

iW3- 2 iW = e iif(h;o)^ u Uw 2 f[z c 

i=2 \ 1 i=l / i a ,...,//>0i=2 V i=l 

From (|1.27[) we obtain 

F(z[H p ];q,t) = J] ? 

/ 

x E E E E n^-w- )^ 

Z>0 <0i ^ 4 fe 2 ,... )fe/ >0 £....^*o <=2 

This immediately implies Q3.12p . 

□ 




4 Proof by Gasper's formula 

Now we are in position to prove Okada's conjecture for birds and banners, i.e., Theorem 11.91 At 
the last step of our proof Gasper's identity (|1.2[) plays an important role. 

We use the fact that Macdonald's polynomials are basis for Ap. (cf. [5]). To prove the birds 
case, we fix integers po and 9q such that 9q > po > 0, and nonnegative integers r\, . . . ,rf. If we 
compare the coefficient of n£=i^T ' PxfiiZai > xiz a2 ; q, t)Qx(yp 1 , yp 2 ; q, t) in p.8[) and (|3.12[) . the 
following identity must hold: 

e ^ (?;,,*)=£ e V tPD 7 )( rt t) n/fa°)/ft+^ )' 

(pi P/ ) Z=0 ii (/>o °(eo,Po)l9> r J i=1 

0<P/<--<Pl<PO i i H hl/=l 

where (f?i, . . . , Of) is determined from 9q and (pi, . . . , pf) by using the equations ^ = p.j_i + + 
r, — /9j for i = 1, . . . , /. Since (|2.1[) implies 

Vo.po) = /(«o - Po;0) f {9 ° ;1) f( Po ;0), 
j lyo — po; J- J 

we obtain 

fyo-Lpo-ofa,*) = f(po-l;0)f(9 -l;l) 
b(e , P0 )(q,t) /(p o ;O)/(0 o ;l) 

Hence it is enough to prove 

e e /( %:;oS/!ti)' :1) n /ft;0)/(i - + " ^ ( "» 

(pi. - .p/) i=0 ii,...,i/>o u ' ' i=l 

o<p/<-<pi<po ! :H H/=i 

In the case of banners we fix a partition A = (Ai, A2, A3, A4) of length 4 and nonnegative integers 

r2, • • • , Tf. If we compare the coefficient of nf=2 ' ^\(^i2q:i , £i2a 2 \ Qi t) m P-10p and (|3.13l) . the 
following identity must hold: 

(P2. ■■■./>/) ;=o i 2 ,- -,'/>o a vy> j i=2 

0<P/<-<P2<Pl ! 2H H/=l 
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where (0 2 ,.. .,6/) is determined from 9\ and (p 2 , • • • , Pf) by using the equations 6i = pi-\ + + 
Ti — pi for i = 2, . . . , /. Here we use the convention that p\ = A4 and 9\ = A 2 . Again, because of 
(|2.2[) we obtain 

£»A-M<(g.*) = /(A 4 -/;0)/(A 2 -f;2) 
bf(q,t) /(A 4 ;0)/(A 2 ;2) ' 

Hence it is enough to prove 

e $((^;^)=e e /(A ; ( A!:o!fSt2) ;2) n / ^ ;o ^^ + ^°^ ^ 

(P2.--.P/) i = l 2 ,--,'/>0 U\ ■il ) i=2 

0<Pf<-<p 2 <Pl <2H M/=l 

In fact a more general formula holds. If we prove the following theorem, then the proof of (|4.1I) 
and (|4.2p are both done. 

Theorem 4.1. Let m and n be nonnegative integers. Let fco, po, #0 be integers such that < 
ko < po < Oq, and let 71, . . . , 7„ be nonnegative integers. Then we have 



n 

i=l 



f(p n - ^o; O)/(0 n - k Q ; m + n) 

k Q<Pn<--<P\<PQ 

f{pi-i - Pi;0)f(0j-i - pi\i + m- l)/(flj - pj-ijj + to - l)/(6> t - 6>,_i;0) 
/(0i -/£>,•;* + m - - Pi! « + TO ) 

n n n 

E /(po-£*i;0)/(flo-£*i;m)n/(*i;0)/(*i + 7i;0), (4.3) 

*1 <=n>o i = 1=0 i=l 

fclH K^Ti<P0~Pm+l 

where the sum on the left-hand side runs over all n-tuples (pi, . . . , p n ) of nonnegative integers such 
that ko < p n < ■ ■ ■ < pi < po, the sum on the right-hand side runs over all n-tuples (fci, . . . , k n ) of 
nonnegative integers which satisfy k\ + • • • + k n < po — p m +i, and 8i is determined from p i; pu—i 
and Oi-i by 9 t = 7* + + pi_ x - p t for i = 1, . . . , n. 

Before we prove this theorem, we need the following lemma which is a special case (i.e., n = 1) 
of this theorem. 

Lemma 4.2. Let m be a nonnegative integer. Let fco, Po and #o be integers such that < fco < 
Po < $0j and let 7 be a nonnegative integer. Then we have 

> / {p - fc ; O)/(0 - fc ; m + 1 j- r—^ — 

j f{v - p;m)f{6 - p;m+ 1) 

Po— &0 

= E /(Po-fco-fc;O)/(0 o -fco-fc;m)/(fc;O)/(fc + 7 ;O), (4.4) 

fc=0 

where 6* = 7 + po + #0 — P- 

Proof. Set 5i to be the left-hand side of (|4.4p . If one puts fc = po — P, then p = po — fc and 
= fc + 7 + 60 ■ Hence one obtains 

Po— fco 

Si = E /fo> - fc ° - fc ; °)/ ( fc + 7 + ^0 - fc ; to + 1) 

fc=0 

/(fc; 0)/(fc + 7 + 6p - po; TO)/(fc + 0Q- po; m)f(k + 7; 0) 
f(2k + 7 + do - Po; m)/(2fc + 7 + O - p ; to + 1) 

If we use 

{a;qhk = {a? ; q) k {-az ; q) k (a>2 ; q) k {-ai ; q) k , 
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then the factors in the denominator are written as f(2k + 7 + 6*0 — po; m) = f{j + Oo — po; m) x 

/ m+1 -y + Op -PQ m + 1 7 + m+1 7 + g O -PQ + 1 m + 1 7+gfl -PQ + 1 \ 

(fTg 2 ,-iT"g 2 .t - S - q 2 ,-t — 2~ (J 2 ;q j 

; -,+o - PO+ i ~ -,+e - P0+ i m T+80-P0+ 2 ^+ 8o -p 0+ 2 \ k and /(2fc + 7 + 6» -po;m+l) = 

i 2 q 2 ,-t 2 q 2 ,t2g 2 ,-t 2 q 2 ; q 



/(7 + o ~ po;m + l) x 

m + 2 -t + 6 n -p n tti + 2 T + 8p-Pn m + 2 -y + 8 n -p n + l m + 2 T + 8 n -Pn + l 

t 2 q 2 ,—4 2 q 2 ,t 2 q 2 2 q 2 ;q 

,+i ^+8 -po+i m+ i ^+a -p +i m+ i ^+^- -7^+2 m+1 7+g0 -p + / V- Meanwhile, the factors in 

"S - q 2 2 (J 2^ .t - S - q 2 ,-t - 2~ q 2 ;g j 

/ k 

the numerator arc f(p -k -k; 0) = /(p - fco; 0) (t Ji 9 g - ° + ° +i^) fc (t) > /(fc + 7+0o - fco;ro + l) = 

/(7 + 6> - fc ; m + 1) ^1^?^^ > /(& + 7 + #o - Po; m) = /(7 + 8 Q - p ; m)^4w^r^, 

f(k+9 -p ;m) = f(e -poim) ^fZZ°i'.$ k k , f(k+ T ,0) = f(k+ r ,0) Hence, substituting 

these factors, we obtain 



S x =C ■ 12 W n ( bc/d; (bcq/ad)^,-(bcq/ad)^,q(bc/d)^,-q(bc/d)^, 
ab/d, ac/d, a, b, c; q, q/a) , 
where a = t,b = tqf, c = q-P°+ k >\ d = t- m q- e °+ k ° and 

c = /(Po - fco; 0)/(7 + ^0 - fco; w + 1)/(0 O - Po; m)/( 7 ; 0) 
/( 7 + O - Po,m + 1) 

On the other hand, Set S2 to be the right-hand side of (|4.4p . If we use /(po — fco — k;0) = 
f(po - fco;0) (t -^W^ )fc (I)', /(0o ~ fco - fc;m) = /(0 O - k ; m ) { X::f::Z^q )k (!)* and 



/(A + 7; 0) = f(k + 7; 0)^^, then wc 



obtain 



S2 = f(Po - fco;O)/(0 o - fco; m)/(7; 0)4^3 



£2 



Hence Gasper's formula (|1.2[) proves that Si = S < 2. The details are left to the reader. This 
completes our proof. □ 

Proof of Theorem 14.11 We proceed by induction on n. If n = 1, then (|4.3[) is nothing but (|4.4ll . 

Let n > 2 and assume (|4.3p is true for n — 1. If wc set S to be the left-hand side of (|4.3p . then we 
have 

^ /(0 1 -pi;m)/(0i-pi;m + l) 



X 



n 



E /(Pn - fco;O)/(0„-fco;m + n) 

(P2 1 ■ ■ ■ iPn) 
fc O<P?i<--<P2<Pl 

/(Pi-i - Pi;O)/(0i-i - pui + m- l)/(0i - + m - l)/(0j - 6>,-i;0) 



/(0i -pi;i + m- l)/(0i -p,-;j + m) 



We can use our induction hypothesis to obtain 

n 

5= E I/(*i;0)/(fci + 7i;t>) 



fc3,...,fcn>0 i=2 
fc 2 _| \rk n <p Q -k Q 

po n n 

x £ /(pi-fco-^fci;O)/(0i-fco-^fc i; m+l) 

/(po - Pi; O)/(0 O - pi;m)/(0i - po; m)/(0i - O ; 0) 



/(0i -pi;m)/(0i -pi;m+l) 
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If we use (|4.4p again, then we obtain 



s= I/(*i;o)/(fti + 7<;0) 



fe 2 ,...,fen>0 i = 2 
^2H \- k n <Po~ k a 

n n 

x E /(po-SAi.Oj/Cflo-S^mJ/CAx.Oj/Cfci+Tl.O), 

0<fcl</do-feo-S" =2 fc i i=0 i = 

which equals the right-hand side of (|4.3[) . This completes our proof. □ 

Concluding Remarks In the proof of the (q,t) hook formula for birds and banners, Gasper's 
identity (|1.2[) for 12 Wn plays an important role. The author tried the other classes of irreducible 
d-complcte posets, but it seems that another identity will be needed for the rest. 
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